Determining camera auto-focus settings

ABSTRACT

A system and method of determining a tilt angle of a portable computing device using sensor data; identifying a tilt angle from a plurality of predetermined tilt angle ranges; determining focal length settings for image capture devices of the portable computing device using the tilt angle, adjustment increments, and autofocus scan range algorithms. A portable computing device including a processor; a first image capture device on a first side of the portable computing device, and a second image capture device on the second side of the portable computing device, the second side located opposite of the first side; and a memory device including instructions operable to be executed by the processor to perform a set of actions, enabling the portable computing device to perform the method.

BACKGROUND

People are increasingly utilizing portable computing devices to takepictures. High quality, high resolution images may be captured with aportable computing device equipped with a lens system, such as a movablelens system. The lens system may be controlled by an automatic lensfocusing system, also referred to as an autofocus system. An autofocussystem may determine one or more focus settings that may include anappropriate lens position or configuration that will result in a sharpand clearly focused image of an object or a situation. It is desirableto obtain the appropriate focus settings quickly in portable computingdevices to conserve battery power, processing capacity, and to provide agood user experience that allows the user to quickly capture awell-focused image. While autofocus techniques exist, many are slow,wasting power, processing capability, and time. Other autofocustechniques may not adequately focus the lens quickly enough oraccurately enough to provide a high quality, high resolution image.

Further, in order to obtain desirable images, portable computing devicesmay utilize an autofocus scan range algorithm to adjust the lens of thecamera(s) capturing the image(s). In conventional approaches, theautofocus scan range algorithm may operate through several iterations toobtain a satisfactory setting for the present image conditions. Eachiteration may use precious time, power, and processing capability of acomputing device. The latency of each iteration may delay a user'sability to capture an image of an object or a situation subject tochange or movement over time. The operation of multiple iterationsconsumes battery power and processing capability which may impact theoverall performance of the portable computing device. Such factors mayresult in a poor user experience. A need, therefore, exists foroptimized autofocus systems.

BRIEF DESCRIPTION OF THE DRAWINGS

Various embodiments in accordance with the present disclosure will bedescribed with reference to the drawings, in which:

FIGS. 1A-1G are schematic diagrams, illustrating examples of a usercapturing an image using a portable computing device;

FIGS. 2A and 2B are schematic diagrams, illustrating examples ofsituations of a portable computing device capturing images of objects;

FIGS. 3A-3D are flow chart diagrams, illustrating exemplary processesfor determining an autofocus scan range algorithm of a camera that maybe used in accordance with various embodiments;

FIGS. 4A-4B are flow chart diagrams, illustrating exemplary processesfor determining an autofocus scan range algorithm of a camera that maybe used in accordance with various embodiments;

FIG. 5 is a schematic block diagram illustrating an exemplaryconfiguration of components of a portable computing device such asillustrated in FIGS. 1A-1G;

FIG. 6 is a schematic diagram illustrating an exemplary environment inwhich various embodiments may be implemented;

FIG. 7 is a schematic diagram illustrating an exemplary environment inwhich various embodiments may be implemented;

FIG. 8 is a schematic diagram illustrating an exemplary process forjoining images using a panoramic camera system in accordance withvarious embodiments;

FIG. 9 is a schematic diagram illustrating an exemplary situation of amobile portable computing device in accordance with various embodiments;

FIG. 10 is a schematic diagram illustrating an exemplary situation of amobile portable computing device in accordance with various embodiments;

FIG. 11 is a chart illustrating the relationship between focused imagequality and focal length in determining the autofocus setting inaccordance with various embodiments;

FIG. 12 is a schematic diagram illustrating an exemplary situation of amobile portable computing device in accordance with various embodiments;

FIG. 13 a schematic diagram illustrating an exemplary situation of amobile portable computing device in accordance with various embodiments;

FIG. 14 a schematic diagram illustrating an exemplary situation of amobile portable computing device in accordance with various embodiments;

FIG. 15 is a flow chart diagram, illustrating an exemplary process fordetermining a first focal length setting and optionally a second focallength setting for an image capture device in accordance with variousembodiments; and

FIG. 16 is a flow chart diagram, illustrating an exemplary process fordetermining one or more focal length settings for a plurality of imagecapture devices in accordance with various embodiments.

DETAILED DESCRIPTION

Systems and methods in accordance with various embodiments of thepresent disclosure may overcome one or more of the aforementioned andother deficiencies experienced in conventional approaches to capturingimages in an electronic environment. In particular, various approachesprovide for the determining and updating of one or more settings of acamera of a portable computing device using sensor information (e.g.,tilt angle, distance, light, etc.) from one or more other sensors of theportable computing device.

In certain embodiments, approaches in accordance with variousembodiments attempt to determine a tilt angle or an orientation of aportable computing device. More specifically, such embodiments maydetermine the title angle or an orientation of a surface of a lens of acamera of a portable computing device. Typically, but not necessarily,the surface of the lens of a camera of a portable computing device isparallel with, or even flush with, a surface of the portable computingdevice. In which case, the tilt angle or orientation of the portablecomputing device is approximately equivalent to the title angle ororientation of the surface of a lens of the camera. Such embodiments mayuse the tilt angle or the orientation of the portable computing deviceto select an autofocus scan range algorithm, adjust the camera lensposition based on the autofocus scan range algorithm, and capture animage of the object with the camera of the portable computing device.

For example, in at least some embodiments, a portable computing devicewith at least one camera may be directed at an object. The portablecomputing device may be oriented in a particular way, i.e., at a tiltangle, relative to the object, relative to an arbitrarily defined axis,or relative to gravity. An intuitive and useful axis relative to which atilt angle may be specified may be an axis that is perpendicular to thedirection of gravity. Such an axis is generally parallel to a flat,level surface, making the axis a convenient and intuitive referencepoint from which to measure the tilt angle of a portable computingdevice, particularly a handheld portable computing device. As usedherein, the term “the horizontal” or “the horizontal axis” refers to anaxis that is perpendicular to the direction of gravity. The portablecomputing device may determine the tilt angle relative to gravity orrelative to the horizontal using at least one sensor of the portablecomputing device. Based at least in part on the determined tilt angle,an autofocus scan range algorithm may be selected from a memory of theportable computing device. The lens of the camera may then be adjustedaccording to the selected autofocus scan range algorithm. For exampleand without limitation, a first autofocus scan range algorithm may beused for tilt angles greater than 75 degrees from the horizontal. Asecond autofocus scan range algorithm may be used for tilt angles lessthan 25 degrees from the horizontal. A third auto focus scan rangealgorithm may be used for tilt angles in the range including 25 through75 degrees from the horizontal. The image of the object is captured bythe camera using the selected focus setting. The portable computingdevice may store the captured image and compare the captured image toother stored images utilizing the same or similar autofocus scan rangealgorithms. When a subsequent image is captured using the same orsimilar autofocus scan range algorithm, the portable computing devicemay compare it to previous captured images. If a match is found, theportable computing device may apply the previous autofocus setting tothe camera to quickly capture a high quality, high resolution imagewithout additional determination or processing, conserving resources andimproving image capture time. A given autofocus range algorithm may alsobe adjusted, modified, or optimized based on a determined tilt angle bychanging one or more aspects of the autofocus range algorithm. Forexample, an initial focal length and an adjustment increment may bespecified based on a tilt angle.

In other embodiments, various approaches provide for the determinationof distance of an object to the camera of the portable computing deviceusing an image of an object captured by the camera of the portablecomputing device. For example, in at least some embodiments, using theimage of an object captured by the camera and comparing it to a libraryof recognized, predetermined objects with known dimensions, the portablecomputing device may determine an approximate distance of the camera ofthe portable computing device from the object. For example, the libraryof recognized, predetermined objects may include information such as thesize of a human finger at certain distances from the camera, such as 1cm, 5 cm, or 10 cm captured using the same or similar autofocus scanrange algorithm. Upon recognizing a captured image as an image of ahuman finger, a comparison may be made of various aspects of thecaptured image with one or more items in the library of recognized,predetermined objects. For example, the size of the finger in thecaptured image may be compared to the library of recognized,predetermined objects. Based at least in part on such a comparison, theportable computing device may approximately determine a distance of thehuman finger from the camera of the portable computing device using thesame or similar autofocus scan range algorithm.

In other embodiments, various approaches provide for the determinationof a distance of an object to the portable computing device usingmultiple images of an object captured by the camera of the portablecomputing device. For example, in at least some embodiments, usingmultiple images of an object captured by the camera and comparing themto a library of recognized, predetermined objects with known dimensions,the portable computing device may approximately determine a distance ofthe camera of the portable computing device from the object. Theportable computing device may capture a series of images of the object,over a predetermined period of time, to determine a motion of the objectrelative to the camera or relative to the portable computing device.Autofocus scan range algorithms may be selected or adjusted based on thedetermined relative motion of the object. For example, if the images arethat of a human finger, the camera of the portable computing device maycapture a series of images, over a predetermined period of time, andcompare the captured images to the image(s) of a human finger in theknown library to approximately determine the distance of the humanfinger to the camera of the portable computing device. For example, ifthe captured images show the size of the human finger to be decreasing,the portable computing device may determine that the human finger ismoving away from the camera of the portable computing device, and mayselect a different autofocus scan range algorithm depending upon thecurrent location of the human finger, i.e., the most recent fingercapture image. Alternatively, if the captured images show the size ofthe human finger is relatively constant, the portable computing devicemay determine that the human finger is relatively stationary, and maymaintain the current autofocus scan range algorithm based on the mostrecent finger capture image. If the captured images show that the sizeof the human finger is increasing, the portable computing device maydetermine that the human finger is coming closer to the camera or to theportable computing device, and may select a different autofocus scanrange algorithm depending upon the current location of the human finger,i.e., the most recent finger capture image.

Some embodiments relate to systems that may include a processor; acamera; a sensor that generates sensor data indicating a gravitationalpull on the portable computing device; and a memory device. The memorydevice may include instructions operable to be executed by the processorto perform a set of actions, enabling the system to: determine a tiltangle of the camera using the sensor data; identify a tilt angle rangefrom a plurality of predetermined tilt angle ranges, wherein the tiltangle range overlaps with the tilt angle of the camera; determine afirst focal length setting using a first associative array thatassociates the tilt angle range with the first focal length setting;determine an adjustment increment using a second associative array thatassociates the adjustment increment with the tilt angle range; anddetermine a second focal length setting of the camera using theadjustment increment according to an autofocus scan range algorithm. Asused herein, an associative array, map, symbol table, or dictionary isan abstract data type composed of a collection of (key, value) pairs,such that each possible key appears just once in the collection.

According to some other embodiments, the set of actions may furtherenable the system to: capture image data of an object within a field ofview of the camera; determine an identity of the object by comparing theimage data with image data of known objects; determine a dimension ofthe object by comparing the identity of the object with dimensional dataof known objects; determine a distance from the camera to the object bycomparing the dimension of the object with a scale of the object in thecaptured image data relative to known dimensions of the field of view ofthe camera; use the distance to determine the first focal lengthsetting, wherein the first focal length setting focuses the camera atthe distance; and use the distance to determine the adjustment incrementusing a third associative array that associates the adjustment incrementwith the distance. The set of actions may further enable the system to:identify a touch gesture on a touch screen, the touch gesture being apinch-to-zoom gesture or a tap-to-zoom gesture; and use the touchgesture to determine the first focal length setting and the adjustmentincrement instead of the tilt angle of the camera. The touch-gesturemay, for example, be interpreted as overriding the determination of thefocal length(s) or adjustment increment(s) based on the tilt angle ortilt angle data.

Some embodiments relate to systems that may include a processor; a firstimage capture device on a first side of the portable computing device; asecond image capture device on a second side of the portable computingdevice, wherein the second side opposes the first side; and a memorydevice. The memory device may include instructions operable to beexecuted by the processor to perform a set of actions, enabling thesystem to: determine a tilt angle of the portable computing device usingsensor data; identify a tilt angle range from a plurality ofpredetermined tilt angle ranges, wherein the tilt angle range overlapswith the tilt angle of the portable computing device; determine a firstfocal length setting for the first image capture device using a firstassociative array that associates the tilt angle range with the firstfocal length setting; determine a first adjustment increment for thefirst image capture device using a second associative array thatassociates the tilt angle range with the first adjustment increment;determine a second focal length setting for the first image capturedevice using the first adjustment increment according to a firstautofocus scan range algorithm; determine a third focal length settingfor the second image capture device using a third associative array thatassociates the tilt angle range with the third focal length setting;determine a second adjustment increment for the second image capturedevice using a fourth associative array that associates the tilt anglerange with the second adjustment increment; and determine a fourth focallength setting for the second image capture device using the secondadjustment increment according to a second autofocus scan rangealgorithm. According to some embodiments, the system may further includea sensor that generates the sensor data. The sensor data may indicate agravitational pull on the portable computing device. The set of actionsmay further include enabling the system to: capture image data of anobject within a field of view of a first image capture device; determinean identity of the object by comparing the image data with image data ofknown objects; determine a dimension of the object by comparing theidentity of the object with dimensional data of known objects; determinea distance from the first image capture device to the object bycomparing the dimension of the object with a scale of the object in thecaptured image data relative to known dimensions of the field of view ofthe image capture device; determine the first focal length setting tofocus the first image capture device at the distance; and determine thefirst adjustment increment using a fifth associative array thatassociates the distance with the first adjustment increment.

Some embodiments relate to a portable computing device that may includea processor; a first image capture device or element; and a memorydevice. The memory device may include instructions operable to beexecuted by the processor to perform a set of actions, enabling theportable computing device to: obtain tilt angle data representing a tiltangle of the first image capture device relative to an axis; anddetermine a first focal length setting for the first image capturedevice based at least in part on the tilt angle data. The image capturedevice or image capture element may be any type of camera, any type ofsensor, and combinations thereof. As discussed herein in greater detail,the image capture device or element may include, or be based at least inpart upon any appropriate technology, such as a CCD or CMOS imagecapture element having a determined resolution, focal range, viewablearea, and capture rate.

The instructions may be operable to further enable the portablecomputing device to: determine an adjustment increment for an autofocusscan range algorithm based at least in part on the tilt angle data; anddetermine a second focal length setting for the first image capturedevice by using the adjustment increment according to the autofocus scanrange algorithm. The instructions may be operable to further enable theportable computing device to: determine the first focal length settingbased at least in part on altitude data representing an altitude of theportable computing device.

The instructions may be operable to further enable the portablecomputing device to: assign a first weight factor to the tilt angle; andassign a second weight factor the altitude data, and determine the firstfocal length setting based at least in part on the first weight factorand the second weight factor. The instructions may also be operable tofurther enable the portable computing device to: determine that thealtitude of the portable computing device exceeds a predeterminedaltitude; and assign the second weight factor to be greater than thefirst weight factor.

According to some embodiments, the instructions may be operable tofurther enable the portable computing device to: identify an objectwithin a field of view of the first image capture device; determine adistance from the portable computing device to the object based at leastin part on a known characteristic of the object; and determine the firstfocal length setting based at least in part on the distance. Theinstructions may also be operable to further enable the portablecomputing device to: determine an orientation of the portable computingdevice; and construct a three-dimensional navigation model based atleast on the identity of the object within the field of view of thefirst image capture device and the distance from the portable computingdevice to the object.

The instructions may be operable to further enable the portablecomputing device to determine the first tilt angle data using at leastone of a gyroscope and an accelerometer. The instructions may beoperable to and the portable computing device may be configured toidentify a touch gesture on a touch screen of the portable computingdevice; and wherein the instructions are operable to further enable theportable computing device to determine the first focal length settingbased at least in part on the touch gesture.

Some embodiments relate to a portable computing device that may includea processor; a first image capture device; a second image capturedevice; a memory device. The memory device may include instructionsoperable to be executed by the processor to perform a set of actions,enabling the portable computing device to:

obtain tilt angle data representing a tilt angle of the portablecomputing device relative to an axis; determine a first focal lengthsetting for the first image capture device based at least in part on thetilt angle data; and determine a second focal length setting for thesecond image capture device based at least in part on the tilt angledata. The set of actions may further enable the portable computingdevice to: determine a first adjustment increment for the first imagecapture device based at least in part on the tilt angle data; determinea third focal length setting for the first image capture device by usingthe first adjustment increment according to a first autofocus scan rangealgorithm; determine a second adjustment increment for the second imagecapture device based at least in part on the tilt angle data; anddetermine a fourth focal length setting for the second image capturedevice by using the second adjustment increment according to a secondautofocus scan range algorithm.

According to some embodiments, the portable computing device may furtherinclude at least one of a gyroscope and an accelerometer, and whereinthe set of actions further enable the portable computing device to:determine the tilt angle data using at least one of the gyroscope andthe accelerometer to determine the tilt angle of the portable computingdevice relative to a direction of gravity. The portable computing devicemay further include a touch screen, and wherein the set of actionsfurther enable the portable computing device to: identify a touchgesture on the touch screen; and determine at least one of the firstfocal length setting for the first image capture device and the secondfocal length setting for the image capture device based at least in parton the touch gesture.

According to some embodiments, the set of actions may further enable theportable computing device to: identify a first object within a field ofview of the first

image capture device; determine a distance from the portable computingdevice to the first object based at least in part on a knowncharacteristic of the first object; and use the distance to determinethe first focal length setting, wherein the first focal length settingfocuses the first image capture device at the distance.

According to some embodiments, the set of actions may further enable theportable computing device to: determine the first focal length settingbased at least in part on altitude data representing an altitude of theportable computing device. The set of actions may further enable theportable computing device to: assign a first weight factor to the tiltangle data; and assign a second weight factor to the altitude data, and

determine the autofocus scan range algorithm based at least in part onthe first weight factor and the second weight factor. The set of actionsmay further enable the portable computing device to: determine that thealtitude of the portable computing device exceeds a predeterminedaltitude; and assign the second weight factor to be greater than thefirst weight factor.

The set of actions may further enable the portable computing device to:identify a touch gesture on a touch screen of the portable computingdevice; and determine the first focal length setting based at least inpart on the touch gesture. The touch gesture may be any type of touchgesture, including but not limited to a pinch-to-zoom gesture, atap-to-zoom gesture or a combination thereof.

Some embodiments relate to a method that includes receiving tilt angledata representing a tilt angle of a first image capture device relativeto an axis; and

determining a first focal length setting for the first image capturedevice based at least in part on the tilt angle data. The method mayfurther include determining an adjustment increment for an autofocusscan range algorithm based at least in part on the tilt angle data; anddetermining a second focal length setting for the first image capturedevice by using the adjustment increment according to the autofocus scanrange algorithm. Determining the first focal length setting may,according to some non-limiting embodiments, further based at least inpart on altitude data representing an altitude of the first imagecapture device.

Various embodiments of the method can further involve assigning a firstweight factor to the tilt angle data; and assigning a second weightfactor to the altitude data. According to such embodiments, determiningthe autofocus scan range algorithm may further based at least in part onthe first weight factor and the second weight factor. Other embodimentsmay involve determining that the altitude of the first image capturedevice exceeds a predetermined altitude; and assigning the second weightfactor to be greater than the first weight factor. The method mayfurther include determining the tilt angle of the portable computingdevice using at least one of a gyroscope and accelerometer. The methodmay further include identifying a touch gesture on a touch screen of theportable computing device, wherein determining first focal lengthsetting is further based at least in part on the touch gesture. Someembodiments of the method may further include identifying an objectwithin a field of view of the camera; determining a distance from theportable computing device to the object based at least in part on aknown characteristic of the object; and determining the first focallength setting based at least in part on the distance.

Various embodiments relate to a method that may include obtaining tiltangle data representing a tilt angle of a portable computing devicerelative to an axis, the portable computing device comprising a firstimage capture device and a second image capture device; determining afirst focal length setting for the first image capture device based atleast in part on the tilt angle data; and determining a second focallength setting for the second image capture device based at least inpart on the tilt angle data.

The method may further include determining a first adjustment incrementfor the first image capture device based at least in part on the tiltangle data; determining a third focal length setting for the first imagecapture device by using the first adjustment increment according to afirst autofocus scan range algorithm; determining a second adjustmentincrement for the second image capture device based at least in part onthe tilt angle data; and determining a fourth focal length setting forthe second image capture device by using the second adjustment incrementaccording to a second autofocus scan range algorithm.

Various embodiments of the method may include determining the tilt angledata using at least one of a gyroscope and an accelerometer to determinethe tilt angle of the portable computing device relative to a directionof gravity. The method may include determining the first focal lengthsetting is further based at least in part on altitude data representingan altitude of the portable computing device. The method may furtherinclude assigning a first weight factor to the tilt angle data; andassigning a second weight factor to the altitude data, and whereindetermining the autofocus scan range algorithm is further based at leastin part on the first weight factor and the second weight factor. Themethod may include determining that the altitude of the portablecomputing device exceeds a predetermined altitude; and assigning thesecond weight factor to be greater than the first weight factor. Themethod may further include identifying a touch gesture on a touch screenof the portable computing device, wherein determining the first focallength setting is further based at least in part on the touch gesture.

An algorithm is a formula, a step-by-step set of operations to beperformed. An algorithm is a method that can be expressed in a finiteamount of space and time for calculating a function. Beginning with aninitial state and initial input, the algorithm describes a computation,that when executed, proceeds through a finite number of steps, producingan output and ending at a final state. Essentially, an algorithm solvesa problem.

For example, one algorithm can be used to find the largest number in alist of random numbers. Finding the solution requires examining everynumber in the list. From this follows a simple algorithm, which can bestated as: (1) If there are no numbers in the list then there is nohighest number; (2) Assume the first number in the list is the largestnumber; (3) For each remaining number in the list, if this number islarger than the current largest number, consider this number to be thenew largest number in the list; (4) When there are no numbers left inthe list, consider the current largest number to the largest number inthe list. This type of algorithm can be referred to as a brute forcealgorithm, in that each number in the list is examined individually andseparately, since the overall list of numbers is not known or examinedin advance.

In another example, known as a bubble sort algorithm, the algorithmrepeatedly steps through a list of numbers to be sorted, compares eachpair of adjacent items and swaps them if they are in the wrong order(for example, numerical or alphabetical order). The pass through thelist is repeated until no swaps are needed, which indicates that thelist is sorted. Like the first example, this type of algorithm is also abrute force algorithm.

A famous example of an algorithm is Euclid's Algorithm. This algorithmdetermines the greatest common divisor between two numbers. Assume twonumbers, such as 16 and 12. Divide the first by the second. If there isa remainder (in this example, 4), divide the first number, 16, by thatremainder, 4, which gives you 4 and no remainder. The process iscomplete. The number 4 is the greatest common divisor. This is anexample of a computational algorithm, requiring computations at eachstep. This algorithm differs from the previous two algorithms, which donot require computation, only comparison of two elements.

A final example of an algorithm is a binary search algorithm. Thisalgorithm finds the position of a target value in a sorted array. Thefirst step is to compare the target value to the value of the middleelement in the sorted array. If the target value is equal to the middleelement's value, the position is returned. If the target value issmaller, the search continues on the lower half of the array. If thetarget value is larger, the search continues on the upper half of thearray. The process continues until the element is found and itspositioned is returned, or there are no more elements left to search forin the array and a “not found” indicator is returned. This algorithm canbe considered a divide-and-conquer search algorithm, as opposed to abrute force search algorithm as the first two examples

Various other applications, processes, and uses are presented below withrespect to the various embodiments.

FIG. 1A is a schematic diagram, illustrating an exemplary situation 110wherein a user 104 is holding a portable computing device 102 at a tiltangle 108 relative to a horizontal axis 101 that is perpendicular to anaxis 103 that is parallel to a direction of gravity. Although a portablecomputing device 102 (e.g., an electronic book reader, smart phone, ortablet computer) is shown, it should be understood that any electronicdevice capable of receiving, determining, and/or processing input may beused in accordance with various embodiments discussed herein, where thedevices may include, for example, desktop computers, notebook computers,personal data assistants, video gaming consoles, television set topboxes, smart televisions, portable media players, and autonomousvehicles, among others.

FIGS. 1A-1G are schematic diagrams, illustrating exemplary situations ofa user capturing an image using a portable computing device. In eachsituation an autofocus scan range algorithm may be selected from aplurality of autofocus scan range algorithms based at least on a tiltangle of camera or of the portable computing device. The tilt angle maybe measured between a face of a lens of the camera to a direction ofgravity or to a horizontal axis that is perpendicular to a direction ofgravity. The plurality of autofocus scan range algorithms may be storedin a memory that is accessible to the portable computing device. Each ofthe plurality of autofocus scan range algorithms may be associated witha range of tilt angles for which the autofocus scan range algorithm islikely to be applicable. For example, when a camera or a portablecomputing device is tilted at an obtuse angle relative to a horizontalaxis that is perpendicular to a direction of gravity, a long-distanceautofocus scan range algorithm may be preferable, because the user maybe more likely to be pointing the camera at a distant object. Similarly,when a camera or a portable computing device is tilted at an acute anglerelative to a horizontal axis that is perpendicular to a direction ofgravity, a near-focus autofocus scan range algorithm may be preferable,because the user may be more likely to be pointing the camera at anearby object. In any case, the selected autofocus scan range algorithmmay be set as the default autofocus scan range algorithm for a currentimage capture or for a future image capture. A focus setting of at leastone camera may then be determined based at least in part on the defaultautofocus scan range algorithm. Additionally or alternatively, one ormore aspects of a given autofocus scan range algorithm may be adjustedbased on the tilt angle. The one or more aspects may, for example,include an initial focal length, or an increment for adjusting the focallength according to a given autofocus scan range algorithm. An initialfocal length parameter of an autofocus scan range algorithm may bescaled based on or in proportion to a tilt angle. Similarly, anincrement for adjusting the focal length for an autofocus scan rangealgorithm may be scaled based on or in proportion to a tilt angle.

As shown in FIG. 1A, the user 104 is pointing the camera 106 at anobject 105. In this example, the user 104 may be pointing the portablecomputing device 102 at an object 105 that is in relatively closeproximity to the camera 106. The portable computing device 102determines the tilt angle 108 relative to the axis 101 through at leastone sensor, such as but not limited to an accelerometer, a gyroscope, analtimeter, or a GPS device. If the portable computing device 102determines that the tilt angle 108 has a certain magnitude, then anautofocus scan range algorithm may be selected. The autofocus scan rangealgorithm may be set as the default autofocus scan range algorithm for acurrent image capture or for a future image capture. A focus setting ofat least one camera may be set based at least in part on the defaultautofocus scan range algorithm. For example, a tilt angle 108 may bedetermined to be 45 degrees from the horizontal axis 101. Based on thedetermined tilt angle, an autofocus scan range algorithm may be selectedor adjusted. In the example shown in FIG. 1A, where the tilt angle isless than 90 degrees with respect to the axis 101 that is perpendicularto the direction of gravity 103, the autofocus scan range algorithm maybe selected or adjusted to optimize autofocusing. For example, when thetilt angle is in a range of from 0 to 90 degrees with respect to theaxis 101, the user may be deemed most likely to be attempting tophotograph an object within a near-field range. A near-field range maybe a range of from about 0 centimeters to about 1 centimeter, from about1 centimeter to about 5 centimeters, or from about 5 centimeters toabout 1 meter. As discussed in greater detail herein, the portablecomputing device 102 may also have object recognition capabilities. Theobject recognition capabilities may be used to determine whether anobject is within a field of view of the camera 106. If no object isrecognized within the field of view of the camera 106, the autofocusscan range algorithm may be selected or adjusted to optimizeautofocusing at a greater distance from the camera 106.

Still referring to FIG. 1A, the camera 106 of the portable computingdevice 102 may capture an image of the object 105 and at least a portionmay be displayed on a display screen of the portable computing device102. In order to obtain more information about the object 105, theportable computing device 102 may access a library of recognized,predetermined images stored in a memory accessible to the portablecomputing device 102. The recognized, predetermined images may containdata concerning the characteristics of the objects in the images. If anobject of a known size is recognized, then a distance from the camera tothe object may be determined. Based on the distance, an autofocus scanrange algorithm may be selected or adjusted.

According to various embodiments, the data may include dimensional datarelative to distances from the camera 106 of the portable computingdevice 102. For example, the data may include a first image of aparticular object at a distance of 1 cm from the camera, a second imageof the object at a distance of 2 cm from the camera, etc. As a morespecific, but non-limiting example, the data may include images of afinger at a plurality of distances from the camera, such as 1 cm, 2 cm,5 cm, and 10 cm. The images of a finger at a plurality of distances fromthe camera may indicate a scale of a typical human finger relative tothe camera 106 of the portable computing device 102 at each of theplurality of distances. Upon capturing an image of an object, andrecognizing the object as a human finger, the portable computing device102 may then compare the image capture of a finger to the data containedin the library. If the image capture of the finger is larger than thelibrary finger image at 10 cm from the camera, then the portablecomputing device 102 determines that the finger in the image capture iscloser than 10 cm from the camera 106.

Additionally or alternatively, the data may include relative dimensionsbetween a plurality of recognizable points on an object. An object maybe recognized and a distance between the object and the camera can bedetermined, if a sufficient number of the recognizable points aredetected in an image of an object. The distance from the camera to theobject may be calculated based on a comparison of the distances betweenthe recognizable points in the image of the object and the relativedimensions stored in the data through a process of triangulation.

FIG. 1B is a schematic diagram, illustrating an exemplary situation 110wherein a user 104 is holding a portable computing device 102vertically, i.e. at a tilt angle that is parallel to a direction ofgravity 103. In this example, the user 104 may be pointing the portablecomputing device 102 at an object (not shown) that is not in closeproximity to the camera 106. The portable computing device 102 maydetermine a tilt angle of the portable computing device 102 relative toa direction of gravity 103 through at least one sensor, such as but notlimited to, an accelerometer, a gyroscope, an altimeter, or a GPSdevice. The portable computing device 102 may determine that the tiltangle is approximately vertical, i.e. aligned with or parallel to adirection of gravity. Based on the determined tilt angle, an autofocusscan range algorithm may be selected or adjusted. In the example shownin FIG. 1B, where the tilt angle is approximately 90 degrees withrespect to the axis 101 that is perpendicular to the direction ofgravity 103, the autofocus scan range algorithm may be selected oradjusted to optimize autofocusing. For example, when the tilt angle isabout 90 degrees with respect to the axis 101, the user may be deemedmost likely to be attempting to photograph an object within a mid-fieldrange. A mid-field range may be a range of from about from about 1 meterto about 3 meters, or from about 3 meters to about 9 meters. Asdiscussed in greater detail herein, the portable computing device 102may also have object recognition capabilities. The object recognitioncapabilities may be used to determine whether an object is within afield of view of the camera 106. If no object is recognized within thefield of view of the camera 106, the autofocus scan range algorithm maybe selected or adjusted to optimize autofocusing at a greater distancefrom the camera 106.

FIG. 1C is a schematic diagram, illustrating and exemplary situation 120wherein a user 104 is holding a portable computing device 102 at tiltangle that is approximately perpendicular to a direction of gravity 103,i.e., parallel to the axis 101 that is perpendicular to a direction ofgravity 103. In this example, the user 104 may be pointing the portablecomputing device 102 at an object (not shown) that is in close proximityto the camera 106. The portable computing device 102 determines the tiltangle of the portable computing device 102 relative to the axis 101 orrelative to a direction of gravity 103 through at least one sensor, suchas but not limited to an accelerometer, a gyroscope, an altimeter, or aGPS device. The portable computing device 102 may determine that thetilt angle is approximately perpendicular to a direction of gravity.

FIG. 1D is a schematic diagram illustrating an exemplary situation 130wherein a user 104 is holding a portable computing device 102approximately horizontal relative to a direction of gravity 103, orparallel to an axis 101, to point the camera 106 at an object 112. Inthis example, the user 104 may be pointing the portable computing device102 at an object 112, such as a business card that is in close proximityto the camera 106. In order to obtain more information about the object112, the portable computing device 102 may access a library ofrecognized, predetermined images stored in a memory accessible to theportable computing device 102. The recognized, predetermined images maycontain data concerning the characteristics of the objects in theimages. If an object of a known size is recognized, then a distance fromthe camera to the object may be determined. Additionally oralternatively, the portable computing device 102 may utilizeapplications or software to identify and perform actions on objects ifthe field of view of the camera 106. For example, an application such asAMAZON FIREFLY® from Amazon Technologies, Inc. allows the portablecomputing device 102 to attempt to identify an object by placing it inthe field of view of the camera 106. If the AMAZON FIREFLY® applicationidentifies the object, the application creates a digital representationcalled a digital entity. A digital entity is made of one or more facets,each of which describes an aspect of the identified object. For example,if the identified object is a book, the digital entity includes a bookfacet, which includes such information as the title, author(s),publisher, release date, and other information. If the book is availablefor sale, the digital entity also contains a product details facet,which includes information relevant to purchasing the book, such asprice, ratings, and item number. Based on the distance or otherinformation about the object, an autofocus scan range algorithm may beselected or adjusted. The portable computing device 102 may determine atilt angle of the portable computing device 102 relative to a directionof gravity through at least one sensor, such as but not limited to anaccelerometer, a gyroscope, an altimeter, or a GPS device. In thisexample, the portable computing device 102 is being held at a tilt anglethat is approximately perpendicular to a direction of gravity. Theportable computing device 102 may determine that the tilt angle isapproximately perpendicular to a direction of gravity.

Based on the determined tilt angle, an autofocus scan range algorithmmay be selected or adjusted. In the example shown in FIG. 1C or FIG. 1D,where the tilt angle is about parallel to the direction of gravity 103,the autofocus scan range algorithm may be selected or adjusted tooptimize autofocusing. For example, when the tilt angle is in a range offrom +/−10 degrees from parallel to a direction of gravity, the user maybe deemed most likely to be attempting to photograph an object within amicro-field range. A micro-field range may be a range of from about 0centimeters to about 1 centimeter, from about 1 centimeter to about 10centimeters, or from about 10 centimeters to about 20 centimeters. Asdiscussed in greater detail herein, the portable computing device 102may also have object recognition capabilities. The object recognitioncapabilities may be used to determine whether an object is within afield of view of the camera 106. If no object is recognized within thefield of view of the camera 106, the autofocus scan range algorithm maybe selected or adjusted to optimize autofocusing at a greater distancefrom the camera 106.

FIG. 1E is a schematic diagram illustrating an exemplary situation 140wherein a user 104 is holding a portable computing device 102 at a tiltangle 108 relative to an axis 103 that is perpendicular to a directionof gravity 101. In this example, the user 104 may be pointing theportable computing device 102 at an object (not shown) that is distantto the camera 106, such as 1 m, 10 m, or 100 m away. As illustrated, inthis example, the tilt angle 108 can be about 120 degrees. The portablecomputing device 102 may determine the tilt angle 108 of the portablecomputing device 102 relative to the axis 103 through at least onesensor, such as but not limited to an accelerometer, a gyroscope, analtimeter, or a GPS device. Based on the determined tilt angle, anautofocus scan range algorithm may be selected or adjusted. In theexample shown in FIG. 1E, where the tilt angle is about greater than 90degrees with respect to the axis 101 that is perpendicular to adirection of gravity 103, the autofocus scan range algorithm may beselected or adjusted to optimize autofocusing. For example, when thetilt angle is in a range of from 90 to 180 degrees with respect to theaxis 101 that is perpendicular to a direction of gravity 103, the usermay be deemed most likely to be attempting to photograph an objectwithin a far-field range. A far-field range may be a range of from about10 meters to about 100 meters, from about 100 meters to a maximum focaldistance of the camera 106. As discussed in greater detail herein, theportable computing device 102 may also have object recognitioncapabilities. The object recognition capabilities may be used todetermine whether an object is within a field of view of the camera 106.If no object is recognized within the field of view of the camera 106,the autofocus scan range algorithm may be selected or adjusted tooptimize autofocusing at a greater distance from the camera 106.

FIG. 1F is a schematic diagram illustrating an exemplary situation 150wherein a user 104 is holding a portable computing device 102. The user104 is using a thumb and a finger to perform a pinch gesture, creating azoom window 114 on display screen 116. In this example, the user 104 isproviding input to the portable computing device 102 through the displayscreen 116. When the user 104 points the camera of the portablecomputing device 102 at a particular object, an image of the particularobject may be visible on the display screen 116. The particular objectmay be among one or more other objects at a variety of distances to thecamera of the portable computing device 102. The user 104 may use apinch gesture to generate a zoom window 114 on the display screen 116over a particular portion thereof. The particular portion may contain,for example, the particular object that the user 104 intends tophotograph. Based on the determined tilt angle and a determined distanceto the particular object within the zoom window 114, an autofocus scanrange algorithm may be selected or adjusted as already described. If atouch gesture is detected on the touch of the portable computing device102, this may indicate that the object is at a distance and the user isattempting to enlarge the image of the object for optimal view,therefore, the detection of the pinch gesture may be a factor indetermining the selection or adjustment of the autofocus scan rangealgorithm.

FIG. 1G is a schematic diagram illustrating an exemplary situation 160wherein a user 104 is holding an object 118 with a thumb and finger infront of a portable computing device 102. The camera 106 of a portablecomputing device 102 is directed at the finger and thumb of the user 104and an unrecognized object 118, visible on display screen 116. In thisexample, the user 104 may be pointing the portable computing device 102at an object close to the camera 106 of the portable computing device102, at a distance, such as 1 cm, 2 cm, or 5 cm away. In a case where atleast one object is not recognized, the portable computing device 102may attempt to determine a distance to the unrecognized object 118 basedon other data or information captured in the image, for example, afinger, a recognized object, or a shadow.

Once the autofocus scan range algorithm has distance to the unrecognizedobject, the lens of the camera 106 of the portable computing device 102may adjust or select an autofocus scan range algorithm based on thedistance to the unrecognized object and/or based on the other data orinformation captured in the image. In this example illustrated in FIG.1G, the camera 106 may identify one or more fingers disposed in closeproximity to the unrecognized object 118. According to variousembodiments it may be assumed that a user's fingers can only be so faraway from the portable computing device 102, therefore, the user may bedeemed most likely to be attempting to photograph an object within anear-field or micro-field range. For example, a range within 1 meter,which may be the typical length of an arm.

FIG. 2A is a schematic diagram illustrating an exemplary situation 200wherein a portable computing device 202 with a camera 206 is directed atan object 208 a distance 210 away. The distance 210 may optionally bedetermined as described in any of the preceding embodiments. The idealfocal length 212 is represented from the camera 206 to the object 208.The ideal focal length 212 may be a focal length at which the object 208is in an optimized focus. The portable computing device 202 determinesthe tilt angle 108 of the portable computing device 202 relative to anaxis 101 perpendicular to a direction of gravity 103 via at least onesensor, such as but not limited to an accelerometer, a gyroscope, analtimeter, or a GPS device. In this example, the portable computingdevice 202 is approximately parallel with a direction of gravity 103. Inother words, the portable computing device 202 has a tilt angle 108 ofabout 90 degrees with respect to an axis 101 that is perpendicular to adirection of gravity 103. An autofocus scan range algorithm may beselected or adjusted based at least in part on the tilt angle, so that afocus setting of the camera 206 is determined. The autofocus scan rangealgorithm may select or scale an initial focal length that is greaterthan or less than the ideal focal length 212 and adjust the initialfocal length by a focal length adjustment increment until arriving atthe ideal focal length 212. The initial focal length and/or the focallength adjustment increment may be selected or scaled based at least inpart on the tilt angle 108 and/or the distance 210. Once the focussetting of the camera 206 has been determined, the camera 206 maycapture at least one image of the object 208.

The camera 206 of the portable computing device 202 may capture at leastone image of the object 208 and at least a portion of the object 208 maybe displayed on a display screen of the portable computing device 202.In order to obtain more information about the object 208, the portablecomputing device 202 may access a library of recognized, predeterminedimages stored in various locations, such as on the portable computingdevice 202, or on a remote server or data storage device. Therecognized, predetermined images may contain data concerning thecharacteristics of the objects in the images. For example, the data mayinclude dimensional data relative to distance 210 from the camera 206 ofthe portable computing device 202. Additional data may includebiographical or technical information, such as product information,price, and user ratings. At least one image of the object 208, such as atree, may be included in the library of recognized, predeterminedimages. The distance data for the object 208 may be for example, 10 m,indicating a scale of a typical tree to the camera 206 of the portablecomputing device 202 at a distance of 10 m. The portable computingdevice 202 may then compare the image capture of a tree to the tree datacontained in the library of the portable computing device 202. If theimage capture of the object 208 is larger than the library tree image,then the portable computing device 202 determines that the tree object208 in the at least one captured image is closer than 10 m from thecamera 206.

FIG. 2B is a schematic diagram of an exemplary situation 220 wherein aportable computing device 202 with a camera 206 is directed at a person228 and a vehicle 230 at distances 232 and 234, respectively. The focallength 236 is represented from the camera 206 to the person 228. Thefocal length 238 is represented from the camera 206 to the vehicle 230.The portable computing device 202 determines the tilt angle of theportable computing device 202 relative to an axis, as described above,such as a direction of gravity 103 or relative to an axis 101perpendicular to a direction of gravity 103. The tilt angle 108 may bedetermined using at least one sensor, such as but not limited to anaccelerometer, a gyroscope, an altimeter, or a GPS device. In thisexample, the portable computing device 202 is approximately vertical,i.e. parallel to a direction of gravity 103. Once the portable computingdevice 202 has determined that the tilt angle is approximately vertical,an autofocus scan range algorithm is selected from a plurality of tiltangle ranges based at least on the tilt angle 108. The autofocus scanrange algorithm may be set as the default autofocus scan range algorithmfor an image capture. Alternatively, the autofocus scan range algorithmmay be used at least momentarily before another autofocus scan rangealgorithm is selected. In either situation, a focus setting of at leastone camera 206 is based at least in part on an autofocus scan rangealgorithm.

In this example, the camera 206 of the portable computing device 202 maycapture an image of the person 228 and the vehicle 230 and at least aportion of the person 228 and the vehicle 230 may be displayed on adisplay screen of the portable computing device 202. In order to obtainmore information about the person 228 and the vehicle 230, the portablecomputing device 202 may access a library of recognized, predeterminedimages stored on the portable computing device 202. The recognized,predetermined images contain data concerning the characteristics of theobjects in the images. For example, the data may include dimensionaldata relative to distances 232 and 234, respectively, from the camera206 of the portable computing device 202. At least one image of theperson 228 and the vehicle 230 may be included in the library ofrecognized, predetermined images. The distance data for the person 228may be for example, 10 m, indicating a scale of a typical person to thecamera 206 of the portable computing device 202 at a distance of 10 m.The portable computing device 202 may then compare the image capture ofthe person 228 to the person data contained in the library ofrecognized, predetermined images of the portable computing device 202.If the image capture of the person 228 is larger than the library personimage, then the portable computing device 202 determines that the person228 in the image capture is closer than 10 m from the camera 206.Similarly, the distance data for the vehicle 230 may be for example, 15m, indicating a scale of a typical vehicle to the camera 206 of theportable computing device 202 at a distance of 15 m. The portablecomputing device 202 may then compare the image capture of the vehicle230 to the vehicle data contained in the library of recognized,predetermined images of the portable computing device 202. If the imagecapture of the vehicle 230 is larger than the library vehicle image,then the portable computing device 202 determines that the vehicle 230in the image capture is closer than 15 m from the camera 206.

Examples of camera focus devices include voice coil motor (VCM),piezo-electric motor (PE), stepper motor, micro-electro-mechanicalsystems (MEMS) and liquid crystal (LQ) devices. Some camera focusdevices such as PE may incorporate a position sensor, typically a Halleffect sensor that may provide an absolute readout of the lens positionthat may be correlated to a particular distance to the plane of focus.Other camera focus devices such as VCM and LQ typically do not have aposition sensor; in these systems the focal distance may be inferredfrom the state of the camera focus device, i.e., the drive current orvoltage. This may require calibration at the time manufacture. In someembodiments, absolute distance is not of great concern, rather therelative distance is desired, for example to determine superposition. Inthis case, indirect measurement of the camera focus distance will notrequire calibration.

The sensitivity of the focal distance determination will be dependent atleast in part on two system properties: the depth of field of the lensand the accuracy and/or precision with which the lens position may bedetermined. The depth of focus is dependent at least in part on themagnification of the lens and on the lens aperture. All of theseproperties are known when the system is designed and may be used todetermine the precision to which focal distance may be determined andoptimized if need be.

In some embodiments, the autofocusing process may start by causing anautofocus processor (AFP) to make a small change in the focusingdistance of a lens of the camera. The AFP may then read the autofocus(AF) sensor to assess whether or not (and if so, by how much) focus hasimproved in an image captured at the focusing distance. Next, using theinformation from the assessment, the AFP may set the lens of the camerato a new focusing distance (e.g., a new focus setting) and captureanother image at the new focusing distance. The AFP may iterativelyrepeat these steps until satisfactory focus has been achieved.

The autofocusing process may usually finish within a short period oftime, such as a fraction of a second. The autofocus process results indetermining a focus setting of the camera (e.g., a focus distance of thelens of the camera) that provides a satisfactory focus and/or sharpnessfor an area of interest in the image to be captured. Typically, theimages captured (temporarily) at the other focus distances during theautofocus process are discarded. However, in according with variousembodiments of the present disclosure, the image captured at the otherfocus distances during the autofocus process need not be discarded, butinstead may be utilized for depth determination.

The camera begins to capture of a plurality of images, beginning at apredetermined start point and ending at a predetermined end point. Theportable computing device determines distance or depth information basedat least in part on the focus setting of the camera. In each image, atleast a portion of at least one object is identified and assigned asharpness score exceeding a sharpness threshold. The sharpness scoreindicates the amount of image detail conveyed dependent at least in parton the focus setting associated with the camera. A position of depth isdetermined associated with at least the identified portion of the atleast one object, in each image captured by the camera, based on thedepth information for each image.

In some embodiments, an object may be determined by the portablecomputing device to be in focus (which may depend on the particularfocus setting of the camera when the image is captured). The portablecomputing device may utilize image processing (e.g., edge detection,line detection, feature detection, etc.) to determine that the object inthe image is substantially sharp. In other words, the portable computingdevice may utilize image processing to determine that the object in theimage has a sharpness level that exceeds a sharpness threshold. In someembodiments, the sharpness level may be a sharpness score and thesharpness threshold may be sharpness score threshold.

Based on the state of the camera such as the focus distance of the lensof the camera and/or the magnification setting of the camera when theimage is captured, the portable computing device may determine that theobject of the focus in the image has a depth that is substantially faraway from the camera; the depth may be a relative depth with respect toat least another image (e.g., relative to another object(s) of focus inanother image(s)). In some embodiments, at least two images are capturedwith different camera focus settings.

The portable computing device may determine depth information associatedwith the image based on the state of the camera when the image iscaptured. For example, based on the focus setting (e.g., focus distanceof the lens) of the camera, the magnification setting of the camera,and/or state information associated with the camera when the image iscaptured, the portable computing device may determine depth informationassociated with the image (and/or object(s) of focus in the image).

The portable computing device may determine the object(s) of focus inthe image, wherein the object(s) of focus is dependent at least in parton the focus setting of the camera when the image is captured. Forexample, the portable computing device may determine that the objects offocus for the image are two objects that are relatively close together.The portable computing device may utilize image processing to determinethat the objects are substantially in focus (e.g., substantially sharp)as compared to the rest of the image, which is out of focus (e.g.,blurry). The portable computing device may determine depth informationassociated with the image (i.e., associated with the objects of focus inthe image).

In some embodiments, the objects in the plurality of the images mayoptionally be “placed” on (e.g., mapped to, associated with, etc.) aplurality of virtual depth planes. In some embodiments, the portablecomputing device may optionally generate a plurality of virtual depthplanes such that each plane representing an area at a particulardistance in depth away from the camera. The number of virtual depthplanes does not have to equal the number of images; there may be more orless virtual depth planes than images. The computing device mayassociate (e.g., “place”) each of the objects in the plurality of imageswith an appropriate virtual depth plane depending on the depthinformation (e.g., depth rank, depth position) associated with eachobject. In some cases, multiple objects may be placed/mapped onto(associated with) a single virtual depth plane. In some cases, somevirtual depth plane may include (be associated with) no objects. In someembodiments, an object (i.e., or a portion thereof) may be associatedwith a virtual plane by tagging the object (or portion) with informationthat indicates that the object (or portion) is virtually located at thevirtual plane.

In some embodiments, the virtual depth planes may be utilized to sortthe plurality of images and/or the objects in the plurality of images.In some embodiments, the sorting may occur priorly and the generating ofthe virtual depth planes may depend on the sorting. In some embodiments,the virtual depth planes may be utilized to build a depth map for theimages and/or the objects in the images.

In some embodiments, the objects of focus from the plurality of imagesmay be placed together to form an image/rendering (i.e., graphicalrepresentation) in which all of these objects are in focus. For example,in some embodiments, all of the objects in the image/rending may be infocus and/or have a sharpness level above a sharpness threshold.

The portable computing device may recognize one or more objects,utilizing object recognition technology (including facial detectionand/or recognition), within a field of view of the camera of the device.For the recognized objects, the portable computing device may provideinformation about the objects. In some embodiments, the portablecomputing device may render/display graphical elements on a displayscreen in conjunction with the objects within the field of view of thecamera. Each graphical element may provide information (or access toinformation) about an object with which the respective element isassociated. For example, an element associated with an object mayprovide a link to information about the object that may be accessed by auser interaction (e.g., tapping on the link).

In some embodiments, the elements may be rendered in accordance with thedepths associated with the objects. Rendering multiple elements on thedisplay screen of the portable computing device may lead to clutterand/or an inconvenient user interface. As such, based at least in parton the depths associated with the objects, the rendering of the elementsmay be adjusted. For example, elements associated with objects in thefront may superimpose and be larger than elements associated withobjects in the back.

It should be understood that there may be additional, fewer, oralternative steps performed in similar or alternative orders, or inparallel, within the scope of the various embodiments unless otherwisestated. In some embodiments, the example method embodiment may startwith capturing images during an autofocus of the camera, each imagebeing captured with a different camera focus setting. The example methodmay also determine a virtual depth plane (i.e., virtual plane) for eachimage based on a focus distance of a camera lens when the image iscaptured.

In some embodiments, the portable computing device may rank theplurality of images (i.e., rank the objects in the plurality of images)based on depth. For example, the portable computing device may determinethat out of the plurality of images captured, the object(s) of focus inthe image associated with the farthest depth may have the highest depthrank, such as being first in a sequential order that ranks the objectsin the images from farthest away from the camera to the closet. Theobject(s) of focus in the image associated with the next farthest depthmay have the next highest depth rank. Finally, in this example, theclosest object may have the last depth rank.

In some embodiments, the portable computing device may generate a depthmap which indicates relative depths for the objects in the plurality ofimages. For example, the depth map may indicate locations of the objectsin a field of view of the camera and indicating relative positions ofdepth for the objects. In some embodiments, the depth map may begenerated based in part on comparing positions in a sequential order ofdepth (e.g., depth ranks) for the objects. In some embodiments, thedepth map may be generated for a plurality of fields of view of thecamera joined together.

In some embodiments, one or more orientation sensors of the portablecomputing device may be utilized at least in part to determine adirection at which the camera of the portable computing device ispointed. The generating of the depth map may be based at least in parton the direction at which the camera is pointed. The one or moreorientation sensors may include (but is not limited to) anaccelerometer, a gyroscope, an electronic compass, or a magnetometer.

In some embodiments, if a depth cannot be determined for an object, thenthe depth for the object may be estimated from the depth of a nearbyobject. For example, if the confidence of the depth determination for anobject is low, then the determining of the depth for the object mayinclude utilizing at least in part a determined a depth for an objectwithin an allowable distance of the former object.

In some embodiments, identifying an object(s), in each of the pluralityof images, that has a sharpness score (i.e., sharpness level) exceedinga sharpness score threshold (i.e., sharpness level threshold, sharpnessthreshold) may be based on at least one of rise distance evaluation,contrast evaluation (e.g., increase resolution and evaluate contrast),fractal dimension evaluation, edge detection, line detection, shadowdetection, feature detection, and/or other suitable image processingalgorithms.

In some embodiments, information about an object may be retrieved via anetwork from a server external to the portable computing device. In someembodiments, at least some of the processing performed by the portablecomputing device may be performed by the server instead.

The portable computing device may include at least one orientationsensor, such as a position and/or movement-determining element. Such asensor may include, for example, an accelerometer or gyroscope operableto detect an orientation and/or change in orientation of the portablecomputing device relative to gravity, as well as small movements of thedevice. An orientation sensor also may include an electronic or digitalcompass, which may indicate a direction (e.g., north or south) in whichthe portable computing device is determined to be pointing (e.g., withrespect to a primary axis or other such aspect). An orientation sensoralso may include or comprise a global positioning system (GPS) orsimilar positioning element operable to determine relative coordinatesfor a position of the portable computing device, as well as informationabout relatively large movements of the portable computing device.Various embodiments may include one or more such elements in anyappropriate combination. As should be understood, the algorithms ormechanisms used for determining relative position, orientation, and/ormovement may depend at least in part upon the selection of elementsavailable to the portable computing device.

In some embodiments, the portable computing device may include one ormore communication elements (not shown), such as a Wi-Fi, Bluetooth, RF,wired, or wireless communication system. The portable computing devicein many embodiments may communicate with a network, such as theInternet, and may be able to communicate with other such devices. Insome embodiments the portable computing device may include at least oneadditional input device able to receive conventional input from a user.This conventional input may include, for example, a push button, touchpad, touch screen, wheel, joystick, keyboard, mouse, keypad, or anyother such device or element whereby a user may input a command to theportable computing device. In some embodiments, however, such a devicemight not include any buttons at all, and might be controlled onlythrough a combination of visual and audio commands, such that a user maycontrol the device without having to be in contact with the portabledevice.

The portable computing device also may include at least one orientationor motion sensor. As discussed, such a sensor may include anaccelerometer or gyroscope operable to detect an orientation and/orchange in orientation relative to gravity, or an electronic or digitalcompass, which may indicate a direction in which the portable computingdevice is determined to be facing. The mechanism(s) also (oralternatively) may include or comprise a global positioning system (GPS)or similar positioning element operable to determine relativecoordinates for a position of the portable computing device, as well asinformation about relatively large movements of the portable computingdevice. The portable computing device may include other elements aswell, such as may enable location determinations through triangulationor another such approach. These mechanisms may communicate with theprocessor of the portable computing device, whereby the portablecomputing device may perform any of a number of actions described orsuggested herein.

As an example, a portable computing device may capture and/or trackvarious information for a user over time. This information may includeany appropriate information, such as location, actions (e.g., sending amessage or creating a document), user behavior (e.g., how often a userperforms a task, the amount of time a user spends on a task, the ways inwhich a user navigates through an interface, etc.), user preferences(e.g., how a user likes to receive information), open applications,submitted requests, received calls, and the like. As discussed above,the information may be stored in such a way that the information islinked or otherwise associated whereby a user may access the informationusing any appropriate dimension or group of dimensions.

FIG. 3A is a flow chart of an exemplary process 300 in accordance withvarious embodiments. In this example, the portable computing deviceselects a first autofocus scan range algorithm based at least in part ona tilt angle relative to an axis, in step 302. The tilt angle isdetermined relative to gravity using at least one sensor, such as butnot limited to an accelerometer, a gyroscope, an altimeter, or a GPSdevice. Once the portable computing device has determined the tilt anglerelative to gravity in step 302, the portable computing devicedetermines a first focus setting of a first camera based at least inpart on the first autofocus scan range algorithm 304.

FIG. 3B is a flow chart of an exemplary process 310 in accordance withvarious embodiments. In this example, the portable computing deviceselects a first autofocus scan range algorithm based at least in part ona tilt angle relative to an axis 302. The tilt angle is determinedrelative to gravity using at least one sensor, such as but not limitedto an accelerometer, a gyroscope, an altimeter, or a GPS device. Oncethe portable computing device has determined the tilt angle relative togravity in step 302, the portable computing device determines a firstfocus setting of a first camera based at least in part on the firstautofocus scan range algorithm 304. The portable computing device thenselects a second autofocus scan range algorithm based at least in parton the tilt angle relative to the axis 306. The tilt angle may be thesame tilt angle as that measured in step 302 or the tilt angle may bedifferent from the earlier measurement, if for example, the portablecomputer device has changed position relative to the axis. The portablecomputing device then determines a second focus setting of a secondcamera based at least in part on the second autofocus scan rangealgorithm 308.

FIG. 3C is a flow chart of an exemplary process 320 in accordance withvarious embodiments. In this example, the portable computing deviceselects a first autofocus scan range algorithm based at least in part ona tilt angle relative to an axis 302. The tilt angle is determinedrelative to gravity using at least one sensor, such as but not limitedto an accelerometer, a gyroscope, an altimeter, or a GPS device. Oncethe portable computing device as determined the tilt angle relative togravity in step 302, the portable computing device determines a firstfocus setting of a first camera based at least in part on the firstautofocus scan range algorithm 304. The portable computing device thenselects a second autofocus scan range algorithm based at least in parton the tilt angle relative to the axis 306. The tilt angle may be thesame tilt angle as that measured in step 302 or the tilt angle may bedifferent from the earlier measurement, if for example, the portablecomputer device has changed position relative to the axis. The portablecomputing device then determines a second focus setting of a secondcamera based at least in part on the second autofocus scan rangealgorithm 308. The portable computing device then identifies an objectwithin a field of view of the camera 312. The portable computing devicecompares the object in the field of view of the camera with a library ofknown objects. The library of known objects may be located in the memoryof the portable computing device, or in other locations, such as but notlimited to the Internet or in a cloud computing platform. Once theobject has been identified, the portable computing device thendetermines a distance from the portable computing device to the objectbased at least in part on a known characteristic of the object, whereinselecting the first autofocus scan range algorithm is further based atleast in part on the distance 314. Examples of known characteristics ofthe object include, but are not limited to, its shape, color, and itsrelationship to other objects.

FIG. 3D is a flow chart of an exemplary process 320 in accordance withvarious embodiments. In this example, the portable computing deviceselects a first autofocus scan range algorithm based at least in part ona tilt angle relative to an axis 302. The tilt angle is determinedrelative to gravity using at least one sensor, such as but not limitedto an accelerometer, a gyroscope, an altimeter, or a GPS device. Oncethe portable computing device as determined the tilt angle relative togravity in step 302, the portable computing device determines a firstfocus setting of a first camera based at least in part on the firstautofocus scan range algorithm 304. The portable computing device thenselects a second autofocus scan range algorithm based at least in parton the tilt angle relative to the axis 306. The tilt angle may be thesame tilt angle as that measured in step 302 or the tilt angle may bedifferent from the earlier measurement, if for example, the portablecomputer device has changed position relative to the axis. The portablecomputing device then determines a second focus setting of a secondcamera based at least in part on the second autofocus scan rangealgorithm 308. The portable computing device then identifies an objectwithin a field of view of the camera 312. Once the object has beenidentified, the portable computing device then determines a distancefrom the portable computing device to the object based at least in parton a known characteristic of the object, wherein selecting the firstautofocus scan range algorithm is further based at least in part on thedistance 314. Using the identification information, the portablecomputing device may determine the dimensions of the object. Theportable computing device then constructs a three-dimensionalnavigational map of the field of view of the camera with respect to theobject using the identity of the object and the distance from theportable computing device and the object 316. The portable computingdevice communicates the three-dimensional navigational map to anavigational system of the portable computing device 318. The portablecomputing system may then navigate with the at least three-dimensionalmap with respect to the object with a propulsion system 322.

The portable computing device may repeat the process of capturing animage of an identifying an object repeatedly and quickly, identifyingadditional objects in the field of view of the camera, determining theirdistance to the camera and their dimensions, and adding that informationto the three-dimensional navigation map. If the field of view of thecamera changes, additional objects not previously in the field of theview of the camera may be identified. By repeatedly identifying objectsin the field of view of the camera, the portable computing device maygain even more information about the identified objects. For example,the portable computing device may determine if the distance between thecamera and the portable computing device changes, i.e., whether theobject is stationary, moving closer to or away from the camera. Theportable computing device may further determine over a period of time,whether a particular object is stationary or not, if so, the frequencyof image capture and identification may be reduced. On the other hand,if the object is determined to not be stationary, the portable computingdevice may increase the frequency of image capture and identification,which may require an additional autofocus scan range algorithm to beselected, depending upon but not limited to the rate of movement of theobject to or from the camera, the capabilities of the camera, includingits lens, optical properties, as well as ambient conditions. Bydetermining the distance, or position, of objects over time, theportable computing device may efficiently allocate resources towardobjects that are moving relative to the camera, or where the camera ismoving relative to the objects, or both.

It should be understood that the camera 106 of the portable computingdevice 102 may be located in a variety of positions relative to theportable computing device 106. In some embodiments, the camera 102 isintegrated into a portable computing device 106, such as a tablet ormobile phone. As such, the camera 106 is not typically removable andmust travel with the portable computing device 102. Also, the field ofview of the camera 106 may be limited by the remainder of the portablecomputing device 106, such as the exterior housing or additionalelements such as mobile phone cases or covers which may further limitthe field of view of the camera 102.

In other embodiments, the camera of the portable computing device may belocated remotely from the portable computing device, while stillmaintaining an electronic connection, such as but not limited to a cableor wireless connection. For example, a portable computing device mayalso include a propulsion system that allows it to move, such as fly.The propulsion system may be of many types, some of which include aplurality of propellers. These propellers are typically located somedistance apart for several reasons, not the least of which is to avoidpropellers contacting one another while rotating and also to providestability. These propellers however could interfere with the field ofview of the camera of the portable computing device, which may belocated centrally within these embodiments. Therefore, the camera may belocated toward the perimeter when equipped with a propulsion system. Thecamera may also include additional movement features and/or lenses thatallow it to have a large field of view, such as but not limited to agimbal system, which may include additional gyro-stabilization.

FIG. 4A is a flow chart of an exemplary process 400 in accordance withvarious embodiments. In this example, the portable computing devicedetermines a tilt angle relative to an axis 402. The axis may be basedon gravity, for example, and determined using at least one sensor, suchan as accelerometer, a gyroscope, an altimeter, or a GPS device. Oncethe portable computing device has determined its tilt angle relative toan axis, the device selects one of a plurality of predetermined tiltangle ranges based on the tilt angle 404. The portable computing devicethen selects an autofocus scan range algorithm based at least in part onthe one of a plurality of predetermined tilt angle ranges 406. Anautofocus scan range algorithm is set as the default autofocus scanrange algorithm for an image capture 408. The portable computing devicethen determines a focus setting of at least one camera based at least inpart on the default autofocus scan range algorithm 412.

FIG. 4B is a flow chart of an exemplary process 410 in accordance withvarious embodiments. In this example, the portable computing devicedetermines a tilt angle relative to an axis 402. The axis may be basedon gravity, for example, and determined using at least one sensor, suchan as accelerometer, a gyroscope, an altimeter, or a GPS device. Oncethe portable computing device has determined its tilt angle relative toan axis, the device selects one of a plurality of predetermined tiltangle ranges based on the tilt angle 404. The portable computing devicethen selects an autofocus scan range algorithm based at least in part onthe one of a plurality of predetermined tilt angle ranges 406. Anautofocus scan range algorithm is set as the default autofocus scanrange algorithm for an image capture 408. The portable computing devicethen determines a focus setting of at least one camera based at least inpart on the default autofocus scan range algorithm 412. An object withina field of view of the camera is identified 414. The portable computingdevice then determines a distance from the device to the object based atleast in part on a known characteristic of the object, wherein selectingthe autofocus scan range algorithm is further based at least in part onthe distance 416.

In order to provide various functionality described herein, FIG. 5illustrates an example set of basic components of a portable computingdevice 500, such as the device 102 described with respect to FIGS.1A-1G. In this example, the device 500 includes at least one centralprocessor 502 for executing instructions that may be stored in at leastone memory device or element 504. The processor determines the properautofocus scan range algorithm based at least upon the tilt angle, andobject information. As would be apparent to one of ordinary skill in theart, the device may include many types of memory, data storage ornon-transitory computer-readable storage media, such as a first datastorage for program instructions for execution by the processor 502, thesame or separate storage may be used for images or data, a removablestorage memory may be available for sharing information with otherdevices, etc. The device 500 typically will include some type of displayelement 506, such as but not limited to a touch screen, electronic ink(e-ink), organic light emitting diode (OLED) or liquid crystal display(LCD), although other types of devices 500 may convey information viaother means, such as through audio speakers. In at least someembodiments, the display element 506 provides for touch or swipe-basedinput using, for example, capacitive or resistive touch technology.

As discussed above, the device 500 in many embodiments will include atleast one image capture element 508, such as but not limited to a camerathat is able to image a user, people, or objects in the vicinity of thedevice 500. The image capture element 508 may include, or be based atleast in part upon any appropriate technology, such as a CCD or CMOSimage capture element having a determined resolution, focal range,viewable area, and capture rate. The image capture element 508 mayinclude at least one IR sensor or detector operable to capture imageinformation for use in determining motion or direction of a user,person, or objects. The example device 500 includes at least one motiondetermining component 512, such as but not limited to a gyroscope, usedto determine motion of the device 500. The device 500 may also include afocus and/or exposure component, such as a local microprocessor and/ormicrocontroller. The focus/exposure determining component(s) may includecode that may be executed to analyze distance information and use anautofocus scan range algorithm or other such approach to determine anappropriate focus setting for the image capture element 508 of thedevice 500. The focus/exposure determining component(s) may furtherinclude code that may be executed to analyze intensity data from a locallight sensor or other such data from one or more sensors and use anautomatic exposure control algorithm or other such approach to determinean appropriate exposure setting for the image capture element 508. Thedevice 500 may also include at least one illumination element 510, asmay include one or more light sources (e.g., white light LEDs, IRemitters, or flashlamps) for providing illumination and/or one or morelight sensors or detectors for detecting ambient light and/or intensity,etc.

The example device 500 may include at least one input device able toreceive input from a user. This input device may include but is notlimited to a push button, touch pad, touch screen, wheel, joystick,keypad, mouse, trackball, or any other such device or element whereby auser may input a command to the device. These input devices may beconnected wirelessly to the device, such as but not limited to wirelessIR, Bluetooth™ or other link as well in some embodiments. In someembodiments, however, such a device might not include any input devicesand may be controlled through a combination of visual (e.g., gesture)and audio (e.g., spoken) commands such that a user may control thedevice without having to be in physical contact with the device.

FIG. 6 is a schematic illustration of an exemplary situation 600 whereina portable computing device 602 with a camera 606 is integrated with anautonomous vehicle, or drone 608. The portable computing device 602determines the tilt angle of the portable computing device 602 relativeto gravity through at least one sensor, such as but not limited to anaccelerometer, a gyroscope, an altimeter, or a GPS device. In thisexample, the portable computing device 602 is integrated or attached todrone 608. The portable computing device 602 may, however, be movablerelative to the drone 608, such as with a motorized gimbal system. Oncethe portable computing device 602 has determined the tilt angle of theportable computing device 602, the tilt angle may be communicated to anavigation system and/or the drone 608. The navigation system maynavigate the drone 608 through a communication link, and/or the drone608 may receive commands externally, or be at least partially remotelycontrolled. The portable computing device 602 may utilize sensors suchas a GPS device to determine its current position, and that of the drone608. The portable computing device 602 may utilize its current positioninformation, along with the tilt angle, to determine an autofocus scanrange algorithm and focus setting for situation 600 in the mannerdescribed above. The camera 606 of the portable computing device 602focuses on the object(s) for image(s) to be captured by the device,according to the determined autofocus scan range algorithm.

FIG. 7 is a schematic illustration of an exemplary situation 700 whereina portable computing device 702 has multiple cameras 706, 708, 710, and712. The portable computing device 702 determines the tilt angle of theportable computing device 702 relative to gravity through at least onesensor, such as but not limited to an accelerometer, a gyroscope, analtimeter, or a GPS device. In this example, the portable computingdevice 702 is located on a table 730, and the tilt angle isapproximately zero degrees. Each camera 706, 708, 710, and 712 has afield of view 714, 716, 718, and 720 respectively. In this embodiment,four people 722, 724, 726, and 728 are seated around the table 730, inthe fields of view of the cameras. In this embodiment, the portablecomputing device includes four cameras, each with a field of view,although the present disclosure is not limited with regard to the numberof cameras that the portable computing device 702 can have.

In one embodiment, the camera 706 may be a primary or controllingcamera. In this embodiment, the portable computing device 702 determinesthe tilt angle and selects or adjusts an autofocus scan range algorithmbased at least in part on one of the tilt angle ranges, as describedabove. For the present situation 700, the tilt angle is approximately 90degrees with respect to a direction of gravity, i.e., in a substantiallyflat or horizontal orientation. The focus setting for the primary orcontrolling camera 706 is determined based on the selected autofocusscan range algorithm. In one embodiment, the focus setting or one ormore focus setting derived therefrom may then be distributed to theother cameras 708, 710, and 712 of the portable computing device 702.Additionally or alternatively, one or more autofocus scan rangealgorithms may be selected or adjusted for one or more of the othercameras 708, 710, and 712 based on the autofocus scan range algorithmselected or adjusted for the primary controlling camera 706. A focussetting for each of the cameras 708, 710, and 712 may then be set basedon the particular autofocus scan range algorithm selected or adjustedfor that camera. Such techniques may avoid having to duplicate theprocessing and computation completed by the portable computing device702 for camera 706. This method saves processing time, image capturetime, and energy of the portable computing device 702. In thisembodiment, portable computing device 702 may be a CENTRO camera thatcaptures multiple images with multiple cameras to create a panoramic 360degree image. In other embodiments, the focus settings of the cameras706, 708, 710, and 712 may each be different, or at least two may havethe same focus setting. If portable computing device 702 rotates aboutits vertical axis (i.e., an axis extending parallel to a direction ofgravity or through the surface of table 730), each camera 706, 708, 710,and 712 would move its field of view in order to capture images ofpeople 722, 724, 726, and 728. As discussed above, using the camera 706to predetermine a focus setting enables the other cameras 708, 710, and712 to be preset, or at least in a better starting position to capture afocused image of the people 722, 724, 726, and 728 as the portablecomputing device 702 rotates.

FIG. 8 illustrates an example situation 800 wherein a plurality ofimages taken by cameras 706, 708, 710, and 712 are joined or “stitched”together using the methodology disclosed herein. In this example, fourimages 802, 804, 806, and 808 are digitally produced and aligned next toeach other by the portable computing device 702. The cameras 706, 708,710, and 712 have overlapping fields of view 714, 716, 718, and 720 andthe resulting images 802, 804, 806, and 808 have overlapping sections810. The portable computing device 702 digitally overlaps, i.e.,“stitches,” and then crops the remainder of the images 802, 804, 806,and 808 to create a single 360 degree panoramic image 830 of people 822,824, 826, and 828. It should be understood that the previous example isillustrative only and in other embodiments the tilt angle, position ofobjects, number of cameras may be different, but the process ofdetermining focus settings, capturing images and joining them togetherto form a 360 degree panoramic image remains the same.

FIG. 9 illustrates an example situation 900 wherein a portable computingdevice 902 with a camera 906 is integrated with an autonomous vehicle ordrone 908. The portable computing device 902 determines the tilt angle904 of the portable computing device 902 relative to a direction ofgravity 903 through at least one sensor, such as but not limited to anaccelerometer, a gyroscope, an altimeter, or a GPS device. In thisexample, the portable computing device 902 is integrated, or attached todrone 908. However, the portable computing device 902 may be movablerelative to the drone 908, such as with a motorized gimbal system. Oncethe portable computing device 902 has determined the tilt angle 904 ofthe portable computing device 902, the information may be communicatedto a navigation system and/or the drone 908. The navigation system inthe portable computing device 902 may navigate the drone 908 through acommunication link, and/or the drone 908 may receive commandsexternally, or be at least partially remotely controlled. The portablecomputing device 902 may utilize sensors such as a GPS device todetermine its current position, and that of the drone 908. The portablecomputing device 902 may utilize its current position information, alongwith the tilt angle 904, to determine an autofocus scan range algorithmfor the situation 900, in the manner described above. The camera 906 ofthe portable computing device 902 focuses on the object(s) for image(s)to be captured by the device.

In one embodiment, the portable computing device 902 may identifyobjects such as the shadows 914, 916 of the drone 908, and determine adistance (or altitude 910, 912) from the portable computing device 902to the object. The portable computing device 902 may store, within inits own memory, or remote memory, known characteristics of the portablecomputing device 902, as well as the drone 908, which is contributing tothe overall shape of shadows 914, 916. In other words, portablecomputing device 902 may recognize the outline of its shape, in the formof shadows 914, 916. This known characteristic is useful for a varietyof reasons, including to further determine distance between objects, analtitude (if the object is on the ground), and to differentiate betweenshadows created by other objects contributing to a cumulative shadow.For example, if the drone 908 picks up a package of unknown dimensionsand carries that package, the package will cast a shadow along with thedrone 908 to form a cumulative shadow. The portable computing device 908may determine the size of the cumulative shadow and may determine thesize of the package beneath drone 908, without that information beingexplicitly supplied. Determining the size of the package of unknowndimensions may be based at least in part on a combination of the size ofthe cumulative shadow, an altitude 910, 912 of the drone 908, and/or adistance from the drone 908 to the cumulative shadow.

The drone 908 is shown in FIG. 9 in two positions; a first position atan altitude 910 casting a shadow 914 on the ground 918; and a secondposition an altitude 912 casting a shadow 916 on the ground 918. Itshould be noted that in other embodiments, portable computing device902, drone 908, and camera 906 may be combined in one device, though forpurposes of explanation only, are shown here separately. The altitude910 is less than the altitude 912 and the shadow 914 is larger than theshadow 916. When the drone is at altitude 910, the portable computingdevice 902 and camera 906 is also at altitude 910. Due to the proximityto the ground 918, a large shadow 914 is cast beneath the drone 908.

As described above, the portable computing device 902 can select oradjust an autofocus scan range algorithm using a determined tilt angleas well as other factors, such as object recognition. A first weightfactor may be assigned to the tilt angle 904. A second weight factor maybe assigned to an identity of an object. The first weight factor and thesecond weight factor may be used to determine an extent to which theautofocus scan range algorithm should be selected or adjusted based onthe tilt angle or based on an identity of an object. In one embodiment,a predetermined or threshold altitude is identified and it is determinedif drone 908 is above or beneath this predetermined threshold altitude.If, for example, drone 908 is below a predetermined threshold altitude,the tilt angle of the portable computing device 902 may be considered tohave more weight than the identity of the object in the determination ofa distance to the object or to selecting or adjusting the autofocus scanrange algorithm. In which case, the first weight factor value for thetilt angle 904 is given a greater weight than second factor attributedto the identity of the object in selecting or adjusting the autofocusscan range algorithm and focus setting to use. When the drone 908 movesto altitude 912, which may be higher than the predetermined thresholdaltitude, the weight factor attributed to tilt angle 904 may be givenless weight than the weight factor attributed to other factors such asthe identity of the object in selecting or adjusting the autofocus scanrange algorithm and focus setting to use.

In other embodiments, when drone 908 is at certain altitudes, only oneof tilt angle or other factor, such as object recognition, need be used,rather than both. For example, when drone 908 is at very high altitudes,the tilt angle 904 of portable computing device 902 may not beconsidered to be significant and, in some embodiments might be soinsignificant that it need not be included in the determination of thedistance to the object to select or adjust the autofocus scan rangealgorithm and focus setting. In these instances, one or more otherfactors, i.e., object recognition, might be the factors used, withoutconsidering tilt angle 904. Alternately, if the altitude of drone 908 isvery low, the tilt angle 904 of portable computing device 902 may be theonly factor used in determining the distance to the object to select oradjust the autofocus scan range algorithm. In other embodiments, insteadof a threshold altitude, an incremental increase or decrease in altitudeof drone 908 can determine a corresponding incremental adjustment in theweights attributed to tilt angle 904, identity of the object asdetermined through object recognition, or any other factor used indetermining the distance to the object to select or adjust the autofocusscan range algorithm and focus setting.

FIG. 10 illustrates an example situation 1000 wherein a portablecomputing device 1002 with a camera 1006 is integrated with anautonomous vehicle or drone 1008. The portable computing device 1002determines the tilt angle 1004 of the portable computing device 1002relative to a direction of gravity 1003 through at least one sensor,such as but not limited to an accelerometer, a gyroscope, an altimeter,or a GPS device. In this example, the portable computing device 1002 isintegrated, or attached to drone 1008. Once the portable computingdevice 1002 has determined the tilt angle 1004 of the portable computingdevice 1002, the information may be communicated to a navigation systemand/or the drone 1008. The navigation system in the portable computingdevice 1002 may navigate the drone 1008 through a communication link,and/or the drone 1008 may receive commands externally, or be at leastpartially remotely controlled. The portable computing device 1002 mayutilize sensors such as a GPS device to determine its current position,and that of the drone 1008. The portable computing device 1002 mayutilize its current position information, along with the tilt angle1004, to determine an autofocus scan range algorithm for the situation1000, in the manner described above. The camera 1006 of the portablecomputing device 1002 focuses on the object(s) for image(s) to becaptured by the device.

In one embodiment, the portable computing device 1002 may identifyobjects such as the shadows 1014, 1016 of the drone 1008, and determinea distance (or altitude 1010, 1012) from the portable computing device1002 to the object. The portable computing device 1002 may store, withinin its own memory, or remote memory, known characteristics of theportable computing device 1002, as well as the drone 1008, which iscontributing to the overall shape of shadows 1014, 1016. In other words,portable computing device 1002 may recognize the outline of its shape,in the form of shadows 1014, 1016. This known characteristic is usefulfor a variety of reasons, including to further determine distancebetween objects, an altitude (if the object is on the ground), and todifferentiate between shadows created by other objects contributing to acumulative shadow. For example, if the drone 1008 is moving over theground 1020 at an altitude 1012, and it approaches a building 1018, thecamera 1006 will capture a new shadow 1014 at an altitude 1010. Theportable computing device 1008 may determine the altitude of the drone1008 based on the change in size of the shadows 1010, 1012.

At altitude 1012, the drone 1008 is relatively high above the ground, atleast compared to altitude 1010, where the drone 1008 is above abuilding 1018. When the drone is at altitude 1012, the tilt angle of theportable computing device 1002 can have a lower weight factor than thealtitude 1012 in relation to the autofocus setting. In other words, whenthe portable computing device is relatively high above the ground, thereare no objects in close proximity to the camera 1006, so the tilt angle1004 has less of a factor on the autofocus setting than the altitude1012. In order to efficiently and quickly determine the autofocussetting at altitude 1012, the portable computing device 1002 selects anautofocus scan range algorithm that is weighted toward distance ratherthan close-up.

At altitude 1010, the opposite is the case. The drone 1008 is relativelylow to the roof of the building 1018. When the drone is at altitude1010, the tilt angle of the portable computing device 1002 has a higherweight factor than the altitude 1012 in relation to the autofocussetting. In other words, when the portable computing device isrelatively low, or close to the ground, there are objects in closeproximity to the camera 1006, so the tilt angle 1004 can have more of afactor on the autofocus setting than the altitude 1010. In order toefficiently and quickly determine the autofocus setting at altitude1012, the portable computing device 1002 selects an autofocus scan rangealgorithm that is weighted toward close-up rather than distance.

FIG. 11 illustrates an exemplary process of an autofocus algorithm. Thegraph 1100 is bounded by a focal length x-axis 1102 and a focused imagequality y-axis 1104. The focal length x-axis 1102 quantifies the focallength of a camera. The focal length may be quantified in terms of unitsof length. The focused image quality y-axis 1104 quantifies the qualityof an image. The quality of an image may be quantified in terms of thesharpness, brightness, clarity, contrast, tint, and/or hue of an image.For photographing a given object or scene, a line 1106 may be plotted toshow the focused image quality of the object or the scene at a varietyof focal lengths.

An autofocus algorithm may operate by selecting a starting focal length1108, which has coordinate values in the focal length x-axis 1102 andthe focused image quality y-axis 1104. The starting focal length 1108may be used to set an initial focal length of a camera. Thecamera/electronic device may capture an image at the starting focallength 1108 and analyze it to determine the focused image quality. Ifthe image meets one or more predefined criteria for focused imagequality, the autofocus algorithm may stop. Otherwise, the autofocusalgorithm may adjust the focal length incrementally by an adjustmentincrement. After each adjustment of the focal length, a focused imagequality may be determined. A plurality of data points 1110 along theline 1106 may be determined and compared.

According to some autofocus scan range algorithms the adjustmentincrement may be greater at larger focal lengths. For example, whenfocusing on an object that is close to a camera, the adjustmentincrements may alter the focal length by very small increments. On theother hand, when focusing on an object that is far from the camera, theadjustments increments may be comparatively large.

While testing the focused image quality at a variety of focal lengths,the autofocus algorithm may determine a peak or a maximum correspondingto focal lengths at which the focused image quality is maximized. Upondiscovering a peak, the autofocus scan range algorithm may first reducethe adjustment increment to provide more resolution of the line 1106,and may eventually stop making further adjustments to the focal lengthupon identifying a maximum. This process of reducing the adjustmentincrement to find a maximum utilizes time, computing resources, andpower. It is possible, however, that a peak is merely a local optimal1112 for image quality and not a global optimal 1114 for image quality.A local optimal 1112 may be a focal length at which the focused imagequality is locally maximized relative to nearby focal lengths, but maybe have a lower focused image quality compared to a global optimal 1114.In other words, the global optimal 1114 may correspond to the idealfocal length for capturing a picture, whereas the local optimal 1112 isless ideal.

It is undesirable for an autofocus scan range algorithm to expend time,power, and computing resources merely to find a local optimal. Finding aglobal optimal 1114 instead of a local optimal 1112 may be influenced bythe starting focal length 1108 and the initial adjustment increment.Various embodiments determine a starting focal length and adjustmentincrement having the greatest probability of identifying a globaloptimal.

FIG. 12 illustrates an example situation 1200 wherein a portablecomputing device 1202 includes at least one camera 1204 and at least onefield of view 1206. In this situation 1200, the portable computingdevice 1202 is mounted to a handlebar 1208 of a bicycle 1210. Thebicycle 1210 is being ridden by a rider 1212. The cameras 1204 areoriented about the perimeter of the portable computing device 1202. Inthis example, one camera 1204 is oriented at the rider 1212, a secondcamera 1204 is oriented to the right of the rider 1212, a third camera1204 is oriented to the left of the rider 1212, and a fourth camera 1204is oriented away from the rider 1212, in the direction of travel of thebicycle 1210. The respective field of views 1206 of the cameras 1204 iscombined to form a panoramic image by the portable computing device 1202as described above. It should be noted that the situation 1200 isexemplary and not limiting. For example, the number of cameras 1204 ofthe portable computing device 1202 may be more or less than the numbershown. Similarly, the portable computing device 1202 may be mounted in avariety of other locations, such as on a helmet 1214 of the rider 1212.

FIG. 13 illustrates an example situation 1300 wherein a portablecomputing device 1302 includes at least one camera 1306 with a field ofview 1304. The portable computing device 1302 is mounted to a drone1308. The drone 1308, as detailed above, may be autonomous,semi-autonomous, operated by remote control, or a combination ofthereof. In the situation 1300, the drone 1308 with the attachedportable computing device 1302 and camera 1306 is positioned above a box1310 with an indicia 1312. The indicia 1312 may be of a variety oftypes, such as but not limited to a two-dimensional bar code, standardtext, or a combination thereof. The indicia 1312 may include informationpertaining to the size, shape, and weight of the box 1310. For example,the indicia 1312 may include information that the box is 6×6×6 inches,weighs 2.0 lbs, and has a destination address of 123. Oak St.,Springfield, USA. The portable computing device 1302 can includesoftware or other components to interpret the indicia 1312 that iswithin the field of view 1304 of the camera 1306. Once the indicia 1312has been interpreted, the portable computing device 1302 can communicatewith the drone 1308 a variety of commands, such as but not limited to,approach the box 1310; pick up the box 1310; and deliver the box 1310 to123. Oak St., Springfield, USA. In another embodiment, the indicia 1312may include information that results in the portable computing device1302 commanding the drone 1308 to bypass the box 1310, such as if thebox is too large or too heavy for the drone 1308 to transport, or if itis not the appropriate box 1310 to be retrieved (or dropped off) by thedrone 1308.

FIG. 14 is an example situation 1400 wherein a portable computing device1402 includes at least one camera 1406 with a field of view 1404. Theportable computing device 1402 is mounted below a drone 1408. The drone1408, as detailed above, may be autonomous, semi-autonomous, operated byremote control, or a combination of thereof. The cameras 1406 areoriented about the perimeter of the portable computing device 1402. Inthis example, one camera 1406 is oriented in one direction, a secondcamera 1406 is oriented in another direction, a third camera 1406 isoriented in yet another direction, and a fourth camera 1406 is orientedin still another direction. The respective field of views 1404 of thecameras 1406 is combined to form a panoramic image by the portablecomputing device 1402 as described above. It should be noted that thesituation 1400 is exemplary and not limiting. For example, the number ofcameras 1406 of the portable computing device 1402 may be more or lessthan the number shown. Similarly, the portable computing device 1402 maybe mounted in a variety of other locations on the drone 1408.

FIG. 15 is a flow chart diagrams illustrating an exemplary process fordetermining a first focal length setting and optionally a second focallength setting for an image capture device in accordance with variousembodiments. As shown in FIG. 15A an exemplary process 1500 may includereceiving tilt angle data representing a tilt angle of a first imagecapture device relative to an axis 1501. The process may further includedetermining a first focal length setting for the first image capturedevice based at least in part on the tilt angle data 1502. The processmay optionally include determining an adjustment increment for anautofocus scan range algorithm based at least in part on the tilt angledata 1503. Finally the process may optionally include determining asecond focal length setting for the first image capture device by usingthe adjustment increment according to the autofocus scan range algorithm1504.

FIG. 16 is a flow chart diagram, illustrating an exemplary process fordetermining focal length settings for a plurality of image capturedevices. The process 1600 may include obtaining tilt angle datarepresenting a tilt angle of a portable computing device relative to anaxis, the portable computing device comprising a first image capturedevice and a second image capture device 1601. The process 1600 mayfurther include determining a first focal length setting for the firstimage capture device based at least in part on the tilt angle data 1602.Finally, the process may include determining a second focal lengthsetting for the second image capture device based at least in part onthe tilt angle data 1603.

The process 1600 may optionally include determining a first adjustmentincrement for the first image capture device based at least in part onthe tilt angle data 1604. The process 1600 may optionally includedetermining a third focal length setting for the first image capturedevice by using the first adjustment increment according to a firstautofocus scan range algorithm 1605. The process may optionally includedetermining a second adjustment increment for the second image capturedevice based at least in part on the tilt angle data 1606. Finally, theprocess may optionally include determining a fourth focal length settingfor the second image capture device by using the second adjustmentincrement according to a second autofocus scan range algorithm 1607.

Storage media and computer readable media for containing code, orportions of code, may include any appropriate media known or used in theart, including storage media and communication media, such as but notlimited to volatile and non-volatile, removable and non-removable mediaimplemented in any method or technology for storage and/or transmissionof information such as computer readable instructions, data structures,program modules or other data, including RAM, ROM, EEPROM, flash memoryor other memory technology, CD-ROM, digital versatile disk (DVD) orother optical storage, magnetic cassettes, magnetic tape, magnetic diskstorage or other magnetic storage devices or any other medium which maybe used to store the desired information and which may be accessed by asystem device. Based on the disclosure and teachings provided herein, aperson of ordinary skill in the art will appreciate other ways and/ormethods to implement the various embodiments.

The specification and drawings are, accordingly, to be regarded in anillustrative rather than a restrictive sense. It will, however, beevident that various modifications and changes may be made thereuntowithout departing from the broader spirit and scope of the invention asset forth in the claims.

What is claimed is:
 1. A system, comprising: a processor; a first image capture device on a first side of the system; a second image capture device on a second side of the system, wherein the second side opposes the first side; a memory device including instructions operable to be executed by the processor to perform a set of actions, enabling the system to: determine a tilt angle of the system using sensor data; identify a tilt angle range from a plurality of predetermined tilt angle ranges, wherein the tilt angle range overlaps with the tilt angle of the system; determine a first focal length setting for the first image capture device using a first associative array that associates the tilt angle range with the first focal length setting; determine a first adjustment increment for the first image capture device using a second associative array that associates the tilt angle range with the first adjustment increment; determine a second focal length setting for the first image capture device using the first adjustment increment according to a first autofocus scan range algorithm; determine a third focal length setting for the second image capture device using a third associative array that associates the tilt angle range with the third focal length setting; determine a second adjustment increment for the second image capture device using a fourth associative array that associates the tilt angle range with the second adjustment increment; and determine a fourth focal length setting for the second image capture device using the second adjustment increment according to a second autofocus scan range algorithm.
 2. The system of claim 1, further comprising a sensor that generates the sensor data, and wherein the sensor data indicates a gravitational pull on the system.
 3. The system of claim 1, the set of actions further comprising enabling the system to: capture first image data including a representation of an object within a field of view of the first image capture device; determine an identity of the object by comparing the first image data with second image data of known objects; determine a dimension of the object by comparing the identity of the object with dimensional data of known objects; determine a distance from the first image capture device to the object by comparing the dimension of the object with a scale of the object represented in the first image data relative to known dimensions of the field of view of the first image capture device; determine the first focal length setting to focus the first image capture device at the distance; and determine the first adjustment increment using a fifth associative array that associates the distance with the first adjustment increment.
 4. A method comprising: obtaining tilt angle data representing a tilt angle of a portable computing device relative to an axis, the portable computing device comprising a first image capture device and a second image capture device; determining a first focal length setting for the first image capture device based at least in part on the tilt angle data; determining a first adjustment increment for the first image capture device based at least in part on the tilt angle data; determining a second focal length setting for the first image capture device by using the first adjustment increment according to a first autofocus scan range algorithm; determining a third focal length setting for the second image capture device based at least in part on the tilt angle data; determining a second adjustment increment for the second image capture device based at least in part on the tilt angle data; and determining a fourth focal length setting for the second image capture device by using the second adjustment increment according a second autofocus scan range algorithm.
 5. The method according to claim 4, further comprising: determining the tilt angle data using at least one of a gyroscope and an accelerometer to determine the tilt angle of the portable computing device relative to a direction of gravity.
 6. The method of claim 4, wherein determining the first focal length setting is further based at least in part on altitude data representing an altitude of the portable computing device.
 7. The method of claim 6, further comprising: assigning a first weight factor to the tilt angle data; and assigning a second weight factor to the altitude data; and determining the first autofocus scan range algorithm based at least in part on the first weight factor and the second weight factor.
 8. The method of claim 7, further comprising: determining that the altitude of the portable computing device exceeds a predetermined altitude; and assigning the second weight factor to be greater than the first weight factor.
 9. The method of claim 4, further comprising identifying a touch gesture on a touch screen of the portable computing device, wherein determining the first focal length setting is further based at least in part on the touch gesture.
 10. A portable computing device, comprising: a processor; a first image capture device; a second image capture device; a memory device including instructions operable to be executed by the processor to perform a set of actions, enabling the portable computing device to: obtain tilt angle data representing a tilt angle of the portable computing device relative to an axis; determine a first focal length setting for the first image capture device based at least in part on the tilt angle data; determine a first adjustment increment for the first image capture device based at least in part on the tilt angle data; determine a second focal length setting for the first image capture device by using the first adjustment increment according to a first autofocus scan range algorithm; determine a third focal length setting for the second image capture device based at least in part on the tilt angle data; determine a second adjustment increment for the second image capture device based at least in part on the tilt angle data; and determine a fourth focal length setting for the second image capture device by using the second adjustment increment according a second autofocus scan range algorithm.
 11. The portable computing device according to claim 10, further comprising at least one of a gyroscope and an accelerometer, and wherein the set of actions further enable the portable computing device to: determine the tilt angle data using at least one of the gyroscope and the accelerometer to determine the tilt angle of the portable computing device relative to a direction of gravity.
 12. The portable computing device according to claim 10, further comprising a touch screen, and wherein the set of actions further enable the portable computing device to: identify a touch gesture on the touch screen; and determine at least one of the first focal length setting for the first image capture device or the second focal length setting for the second image capture device based at least in part on the touch gesture.
 13. The portable computing device according to claim 10 wherein the set of actions further enable the portable computing device to: identify a first object within a field of view of the first image capture device; determine a distance from the portable computing device to the first object based at least in part on a known characteristic of the first object; and analyze the distance to determine the first focal length setting, wherein the first focal length setting focuses the first image capture device at the distance.
 14. The portable computing device according to claim 10 wherein the set of actions further enable the portable computing device to: determine the first focal length setting based at least in part on altitude data representing an altitude of the portable computing device.
 15. The portable computing device according to claim 14 wherein the set of actions further enable the portable computing device to: assign a first weight factor to the tilt angle data; and assign a second weight factor to the altitude data; and determine the first autofocus scan range algorithm based at least in part on the first weight factor and the second weight factor.
 16. The portable computing device according to claim 15, wherein the set of actions further enable the portable computing device to: determine that the altitude of the portable computing device exceeds a predetermined altitude; and assign the second weight factor to be greater than the first weight factor.
 17. The portable computing device according to claim 10 wherein the set of actions further enable the portable computing device to: identify a touch gesture on a touch screen of the portable computing device; and determine the first focal length setting based at least in part on the touch gesture.
 18. The portable computing device of claim 17, wherein the touch gesture is a pinch-to-zoom gesture, a tap-to-zoom gesture or a combination thereof. 